Universality of deep convolutional neural networks

Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, and many other domains. Deep neural network architectures and computational issues have been well studied in machine learning. But there lacks a theoretical foundation for understanding the approximation or generalization ability of deep learning methods generated by the network architectures such as deep convolutional neural networks. Here we show that a deep convolutional neural network (CNN) is universal, meaning that it can be used to approximate any continuous function to an arbitrary accuracy when the depth of the neural network is large enough. This answers an open question in learning theory. Our quantitative estimate, given tightly in terms of the number of free parameters to be computed, verifies the efficiency of deep CNNs in dealing with large dimensional data. Our study also demonstrates the role of convolutions in deep CNNs.     

基于双向循环神经网络的汉语语音识别*

当前基于深度神经网络模型中,虽然其隐含层可设置多层,对复杂问题适应能力强,但每层之间的节点连接是相互独立的,这种结构特性导致了在语音序列中无法利用上下文相关信息来提高识别效果,而传统的循环神经网络虽然做出了改进,但是只能对上文信息进行利用。针对以上问题,该文采用可以同时利用语音序列中上下文相关信息的双向循环神经网络模型与深度神经网络模型相结合,并应用于语音识别。构建具有5层隐含层的模型,其中第3层为双向循环神经网络结构,其他层采用深度神经网络结构。实验结果表明:加入了双向循环神经网络结构的模型与其他模型相比,较好地提高了识别正确率;噪声对双向循环神经网络汉语识别有重要影响,尤其是训练集和测试集附加噪声类型不同时,单一的含噪声语音的训练模型无法适应不同噪声类型的语音识别;调整神经网络模型中隐含层神经元数量后,识别正确率并不是一直随着隐含层中神经元数量的增加而增加,神经元数量数目增加到一定程度后正确率出现了降低的趋势。     

基于贝叶斯网络的船舶港口国监督检查滞留风险研究

针对港口国监督(Port State Control, PSC)检查的复杂性和不确定性, 基于贝叶斯网络理论构建船舶PSC检查滞留风险分析模型. 以东京备忘录(Tokyo MOU)中2014~2017年船舶PSC检查样本数据为基础, 运用R语言bnlearn包进行贝叶斯网络的结构及参数学习. 同时分别执行贝叶斯网络的正向、逆向推理, 定量表示各风险因素与滞留结果之间的相互作用关系, 找出导致船舶滞留的高风险因素, 实现不确定环境下船舶PSC检查滞留风险的全面动态分析. 实证表明, 模型具有较高的精确度, 可为检查人员的滞留决策及航运公司的安全风险管理提供有效依据.     

Approximate Entropy of Brain Network in the Study of Hemispheric Differences

Human brain, a dynamic complex system, can be studied with different approaches, including linear and nonlinear ones. One of the nonlinear approaches widely used in electroencephalographic (EEG) analyses is the entropy, the measurement of disorder in a system. The present study investigates brain networks applying approximate entropy (ApEn) measure for assessing the hemispheric EEG differences; reproducibility and stability of ApEn data across separate recording sessions were evaluated. Twenty healthy adult volunteers were submitted to eyes-closed resting EEG recordings, for 80 recordings. Significant differences in the occipital region, with higher values of entropy in the left hemisphere than in the right one, show that the hemispheres become active with different intensities according to the performed function. Besides, the present methodology proved to be reproducible and stable, when carried out on relatively brief EEG epochs but also at a 1-week distance in a group of 36 subjects. Nonlinear approaches represent an interesting probe to study the dynamics of brain networks. ApEn technique might provide more insight into the pathophysiological processes underlying age-related brain disconnection as well as for monitoring the impact of pharmacological and rehabilitation treatments.     

贝叶斯优化卷积神经网络公共场所异常声识别*

针对公共场所异常声的感知和识别问题,提出一种基于贝叶斯优化卷积神经网络的识别方法。提取声信号的Gammatone倒谱系数、倍频程功率谱、短时能量和谱质心,组合成声信号的特征图。构建卷积神经网络作为分类器,利用递增的卷积核设置和池化操作处理不同尺度的特征。基于贝叶斯优化算法优化卷积神经网络的模型参数,对包括火苗噼啪声、婴儿啼哭声、烟花燃放声、玻璃破碎声和警报声的5种公共场所异常声进行识别。该方法的识别结果与基于不同的特征提取和分类器方案得到的识别结果进行比较,结果表明该方法的识别效果优于其他特征提取和分类器方案的识别效果。最后分析了该方法在不同信噪比噪声干扰下的识别结果,验证了该方法的有效性。     

Global approximation to arbitrary cost functions: A Bayesian approach with application to US banking

This paper proposes and estimates a globally flexible functional form for the cost function, which we call Neural Cost Function (NCF). The proposed specification imposes a priori and satisfies globally all the properties that economic theory dictates. The functional form can be estimated easily using Markov Chain Monte Carlo (MCMC) techniques or standard iterative SURE. We use a large panel of U.S. banks to illustrate our approach. The results are consistent with previous knowledge about the sector and in accordance with mathematical production theory.     

Semiotic Aggregation in Deep Learning

Convolutional neural networks utilize a hierarchy of neural network layers. The statistical aspects of information concentration in successive layers can bring an insight into the feature abstraction process. We analyze the saliency maps of these layers from the perspective of semiotics, also known as the study of signs and sign-using behavior. In computational semiotics, this aggregation operation (known as superization) is accompanied by a decrease of spatial entropy: signs are aggregated into supersign. Using spatial entropy, we compute the information content of the saliency maps and study the superization processes which take place between successive layers of the network. In our experiments, we visualize the superization process and show how the obtained knowledge can be used to explain the neural decision model. In addition, we attempt to optimize the architecture of the neural model employing a semiotic greedy technique. To the extent of our knowledge, this is the first application of computational semiotics in the analysis and interpretation of deep neural networks.     

Current air quality analytics and monitoring: A review

This review summarizes the different tools and concepts that are commonly applied in air quality monitoring. The monitoring of atmosphere is extremely important as the air quality is an important problem for large communities. Main requirements for analytical devices used for monitoring include a long period of autonomic operation and portability. These instruments, however, are often characterized by poor analytical performance. Monitoring networks are the most common tools used for monitoring, so large-scale monitoring programmes are summarized here. Biomonitoring, as a cheap and convenient alternative to traditional sample collection, is becoming more and more popular, although its main drawback is the lack of standard procedures. Telemonitoring is another approach to air monitoring, which offers some interesting opportunities, such as ease of coverage of large or remote areas, constituting a complementary approach to traditional strategies; however, it requires huge costs.